On Gradient Coding With Partial Recovery
نویسندگان
چکیده
We consider a generalization of the gradient coding framework where dataset is divided across $n$ workers and each worker transmits to master node one or more linear combinations gradients over its assigned data subsets. Unlike conventional which requires recover sum all subsets in presence straggler workers, we relax goal computing at least some notation="LaTeX">$\alpha $ fraction gradients. begin by deriving lower bound on computation load any scheme also propose two strategies achieve this bound, albeit cost high communication number partitions can be polynomial . then schemes based cyclic assignment utilize have load. When single combination, prove bounds using partitions. Finally, describe class different intermediate operating points for provide simulation results demonstrate empirical performance our schemes.
منابع مشابه
On partial sparse recovery
We consider the problem of recovering a partially sparse solution of an underdetermined system of linear equations by minimizing the l1-norm of the part of the solution vector which is known to be sparse. Such a problem is closely related to a classical problem in Compressed Sensing where the l1-norm of the whole solution vector is minimized. We introduce analogues of restricted isometry and nu...
متن کاملGradient Coding
We propose a novel coding theoretic framework for mitigating stragglers in distributed learning. We show how carefully replicating data blocks and coding across gradients can provide tolerance to failures and stragglers for synchronous Gradient Descent. We implement our scheme in MPI and show how we compare against baseline architectures in running time and generalization error.
متن کاملOn Depth Recovery from Gradient Vector Fields
Depth recovery from gradient vector fields is required when reconstructing a surface (in three-dimensional space) from its gradients. Such a reconstruction task results, for example, for techniques in computer vision aiming at calculating surface normals (such as shape from shading, photometric stereo, shape from texture, shape from contours and so on). Surprisingly, discrete integration has no...
متن کاملDirty Paper Coding with Partial State Information
A generalization of the problem of dirty paper coding is considered in which (possibly different) noisy versions of the state information, assumed to be i.i.d. Gaussian random variables, are available at the encoder and at the decoder. This paper derives the maximum achievable rate formula for this general problem. This general setup encompasses the cases where the state information is perfectl...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Communications
سال: 2023
ISSN: ['1558-0857', '0090-6778']
DOI: https://doi.org/10.1109/tcomm.2022.3230779